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Two problems of a nonlinear character concerned with random processes are discussed. 
In both cases the processes are assumed to be stationary. 

The first problem is concerned with the representation of a discrete time parameter 
stationary random process as a one-sided function (nonlinear generally) of independent 
random variables and its shifts. This is a representation one might expect if the process is 
purely nondeterministic. Comments are made on the continuous parameter version of this 
problem, indicating that it is likely to be much more difficult and perhaps less important 
from a practical point of view. The second problem is concerned with the harmonic resolu- 
tion of the moments (of degree two or higher) of stationary random processes. The har- 
monic resolution of third order moments (the "bispectrum") is considered in some detail and 
remarks arc made about statistical estimates of the bispectrum. 



1. Introduction 

1.1. Representation of Discrete Time Parameter 
Processes 

Let {X k ; k = Q, ±1, . . .} be a discrete time param- 
eter stationary random process. It will be conven- 
ient to introduce the following notation. Let ©„(A') 
be the Borel field of events generated by X nj X n ~ u . . . 



(B n (X)=®{X k ;k<n}; 



(1) 



@ n (X) is the smallest collection of events containing 
the events 



{X k <x k }, 



k=n, n—1, 



with the x k s any real numbers, that is closed under 
countable union and complementation of events. If 
time n is considered the present, (B n (X) can be 
thought of as the carrier of the information (non- 
linear) from the infinite past to the present. If the 
intersection U(Bn(X) of the Borel fields (B n (X) is 

n 

the trivial Borel field consisting only of the empty 
set (up to a set of probability zero) and its comple- 
ment, we call the process {X k } purely nondeterminis- 
tic. This means that the infinite past contains no 
information about the present. Notice that the 
concept of a purely nondeterministic process is in- 
timately associated with a sense of time direction. 
A process may be purely nondeterministic and yet 
if we look at it with the time direction reversed, it 
may become purely deterministic (the infinite future 
contains all the information about the present). 
This contrasts strongly with the linear (or weak) 
version of the notion of a purely nondeterministic 



real- valued process. The following well-known 
process is easily seen to be purely nondeterministic 
looking forward in time and purely deterministic 
looking backward in time. 1 {X k } is a stationary 
Markov process on the unit interval 0<x<l with 
instantaneous distribution uniform and the following 
simple transition mechanism 



2«£ 



2" 



with probability 



\x with probability \ 



0<x<l. 



(2) 



Let us now discuss the representation problem of 
in t (Test. We wish to find out when one can iind a 
one-sided Borel function j of independent random 
variables {<•*;&=(), ±1, .. .} uniformly distributed 
on [0,1] such that the process {X k } 



X' k =m, fe-i, **-»,...), i=0 f ±l, 



(3) 



has the same probability structure as {X k ;k=0, 
±1, . . .}. One can easily show that such a 
representation cannot hold unless {X k } is purely 
nondeterministic [Rosenblatt, 1959]. In fact, if 
{X k } is a countable state Markov chain the condition 
that the process be purely nondeterministic is neces- 
sary and sufficient for such a representation [Rosen- 
blatt, 1960; Hanson, 1963]. It is a reasonable con- 
jecture that this condition is necessary and sufficient 
for such a representation in the case of a general 
stationary process. As far as I know this has not 
yet been shown to be true. The following result for 
Markov processes (see [Hanson, 1963]) is an interest- 
ing step in this direction. Assume that {X k } is a 
purely nondeterministic Markov process with in- 



i This was pointed out to the author by B. Jamison. 



933 



stantaneous distribution P(-) and transition function 
P(-, •)• Further let there be a measure <£ and events 
A, B with T(A)>0, </>(£) >0 such that 



P{x,B')>4>(B') 



(4) 



for all xeA and all B'czB. Then a representation 
of the desired type exists for the process {X k }. 

The problem we have discussed above can be 
analyzed in greater detail with some benefit. Let 
us allow {£ w } to be any sequence of independent 
identically distributed random variables. For ex- 
ample, the common distribution function might be 
discrete. If 



*;=/(&,&. 



.) 



*=0,±1, 



(5) 



with some given/, and {X' k } has the same probability 
structure as {X k } we shall say as before that we 
have a one-sided representation of the process {X k }. 
Now let L 2 (X;n) be the family of square integrable 
functions measurable with respect to (B n (X). Fur- 
ther let L 2 (X)dL 2 (X',n) be the square integrable func- 
tions measurable with respect to ©(X) = (B m (X) 
that are orthogonal to L 2 (X; n) . We shall say that the 
representation (5) is a one-sided canonical representa- 
tion of X if L 2 (£;n) is orthogonal to L 2 (X')dL 2 (X'',ri). 
Essentially this says that there is no information 
about the present and past of the £ process in the 
future of the X' process. If ©*(£) = (B n (X f ), we 
will call (5) a one-sided properly canonical representa- 
tion of X. In Rosenblatt [1959], it is shown that a 
one-sided properly canonical representation is ex- 
tremely rare for finite state Markov chains. Let 
P=(p tJ ) be the transition probability matrix of the 
chain. Then a necessary (but not sufficient) condi- 
tion for such a representation is that all the row 
probability distributions {p ij) j=\ 1 2, . . .} be the 
same except for permutation. However, a one-sided 
canonical representation is always possible for a 
purely nondeterministic chain (see [Rosenblatt, 1959] 
for details). The impossibility of a properly canoni- 
cal representation, even though a canonical repre- 
sentation is possible, appears to be due to the fact 
that the probability space of the process is not rich 
enough. 

Wiener has discussed one-sided properly canonical 
representations in his interesting book on nonlinear 
problems ([1958] lectures 12 and 13 on coding and 
decoding). Let {X n } be a stationary purely non- 
deterministic process with 



* \%\%n-l) X n-2 



.)=P(X n <x\Xn-l 

^Xn— 1? <A*n— 2 z= %n 



•) (6) 



the conditional distribution function of X n given the 
past of the process relative to n. Wiener states that 
a properly canonical one-sided representation of 
{X n } is possible as long as F(x\x n _i, x n _ 2 , • • •) is 
properly increasingi or almoste very p as tx n ^ u x n _ 2 , 
It is not quite clear what is meant by the term 
properly increasing. However, from the context it 



would appear to be the case that a distribution 
function is properly increasing if it is absolutely 
continuous with a unimodal density function. 
Wiener's conjectured result may very well be valid 
but the way in which he proposes to construct the 
independent random variables £ w in terms of which 
the properly canonical representation is to be set 
up will not generally work. He proposes to set 



?« — F(X n \X n _i, X n 



•) 



(7) 



and the random variables % n will be independent 
uniformly distributed random variables with 



S.ttJeS.CX) 



(8) 



if F(x\x n _ 1} x n _ 2 , . . .) is a continuous function in x 
for almost every past x n _ h x n _ 2 , .... The follow- 
ing example is one in which it is clear that © w (£)?^ 
(B n (X) so that X n cannot be fully reconstructed from 
£», f„_i, . . . with £ n given by (7). 

Let F(x) be an absolutely continuous distribution 
function with unimodal positive continuous density 
function. The process { X n } is taken to be a station- 
ary Markov process with conditional distribution 
function 



F(x\x n -!) ■■ 



(F(x)H2j-l<x n . 1 <2j 

F(x-l)if2j<x n - 1 <2j+l 

j=0, ±1, ±2,... 



r 



(9) 



One could, for example, take F(x)=$(x) with <$>(#) 
the standard gaussian distribution function. In 
such a case the process { X n } will certainly be purely 
nondeterministic. Then, using the construction (7), 



$n=\ 



(F(X n )U2j-l<X n _ 1 <2j 



lF(X n . 1 )if2j<X n „ 1 <2j+l 

i=o, ±i, — (io) 

However, using knowledge of % ki k<n, we can not 
determine whether the greatest integer less than or 
equal to X n is even or odd. 

The problem of obtaining one-sided representations 
of processes amounts to an investigation of the class 
of processes one can obtain from independent random 
variables by one-sided nonlinear filtering. In the 
case of a one-sided properly canonical representa- 
tion, the process is obtained by an invertible non- 
linear filter whose inverse is one sided. 



1.2. Representation of Continuous Time 
Parameter Processes 

N. Weiner was also concerned with the one-sided 
representation of processes in terms of the Brownian 
motion (or Wiener) process in his book on nonlinear 
methods in random theory [1958]. We shall describe 
some recent work of M. Nisio [1961] on aspects of 
this problem. Let { X(t), — oo<t<oo } be a station- 
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ary purely nondeterministic process and elB~ 
{dB(t), — oo<^<^co} be the Wiener random measure. 
Consider a sequence of functions f={f n , n=0, 1, 2,. . . }, 
where each j„ is a symmetric L 2 function of n vari- 

CO 

ables on the domain ( — °°, 0] w . Assume that 2 ^' 

n = 

ll/nll 2 ^ 00 where ||/ n || is the i 2 norm of / n . Givenany 
suchsequence /", we define the corresponding process 
-ar / = {-Y , (O l -oo<e<a>} as 



ra=0 J-oo 



/: 



(11) 



where the nth term is an n -dimensional Wiener integral. 
\i{X'(t)} has the same probability structure as 
{X(t)} we say that (11) is a one-sided representation 
of X in terms of the Brownian motion with f the kernel 
of the representation. As before we say that the 
representation is canonical if L 2 (dB, t)±L 2 (X')6L 2 
(X f , t). The representation is properly canonical 
if L 2 (dB, t)=L 2 (X', t). The terminology on repre- 
sentations used in this paper was introduced by 
M. Nisio in the continuous time context. 

Miss Nisio has shown that a class of one dimen- 
sional diffusion processes has properly canonical one- 
sided representations in terms of Brownian motion. 
What is much more surprising is that she has shown 
that if P(t), — °° <7 <°°, is a Poisson process with 
parameter X then P(t)—P(t—\) ( — <» <p <oo) has 
no canonical one-sided representation in terms of 
Brownian motion. It is a very interesting question 
as to whether there is any one-sided representation 
of P(t)— P(t— 1) in terms of Brownian motion. One 
is inclined to doubt whether this is so on the basis 
of her result. In the case of a discrete time param- 
eter process it seemed reasonable to conjecture that 
a necessary and sufficient condition for a process to 
have a representation is that it be purely nondeter- 
ministic. If there is a representation, it is always 
possible to set it up in terms of independent uni- 
formly distributed (on [0, 1J) random variables 
However, if the conjecture on the impossibility of a 
one-sided representation of P(t)—P(t—1) in terms 
of Brownian motion is valid, it is likely that in order 
to set up one-sided representations of purely non- 
deterministic processes one must consider not only 
representations in terms of Brownian motion but 
also in terms of any differential process. Further 
the appropriate differential process in which to set 
up the representation is most likely determined by 
the local properties of the given process X(f). 

One should mention that a great deal of the dis- 
cussion in Wiener's book on nonlinear methods is 
concerned with the computation of the expectation 
of polynomial forms in a process with a representa- 
tion in terms of Brownian motion. This study has 
been extended by McShane [1962] and more recently 
Sinai and Shiryaev [1963] to processes with repre- 
sentations of a much more general type. 



2. Higher Order Moment Functions and 
Their Spectral Resolutions 

In the case of a weakly stationary process X t with 
mean zero, EX t =0, the second order moments 
EX t X T =r(t—r) exist and it is well known that there 
is a spectral resolution 



!•(*) = 



f °° e m dF(K) 

J -co 



(12) 



where Fis bounded and nondecreasing. Suppose we 
assume the process X t is strictly stationary and that 
the kih moment B|i?'J*<oo for some integer &>2. 
Let the mean EX t again be identically zero. Then 

E(X h . . . X tk ) = m k (t h . . .,t*)=r k (t 2 —t h . . ., t k — U) 

(13) 

exists for all t u . . ., t k and we might hope for a 
spectral representation of r k (r h . . .,r k ^) 

r*(n, . . ., r»- 1 )=J^" . -J e'^idFtQn, . . ., X,_0 (14) 

in terms of a complex-valued function F k (\ h . . ., \ k _{) 
of bounded variation. Blanc-Lapierre and Fortet 
[1953] were among the earliest to discuss such higher 
order spectral representations. Recently Kolmo- 
gorov and Sinai gave examples of stationary proc- 
esses for which E\X t \ k <^co and yet r k has no spectral 
representation in terms of an F k of bounded variation. 
In fact, in Sinai's example [1963] all moments exist. 
Of course, it is still possible that there may be such 
representations with F k of unbounded variation. 

In any case, it is still interesting to consider the 
large class of processes for which such spectral 
representations of higher order moment functions are 
possible. The spectral representation of the third 
order moment function has drawn especial attention 
(see Tukey [1959] for an example). Such an ap- 
proach seems to be of some interest in studying 
certain nonlinear problems in random process theory. 
For a recent paper using estimation of the third order 
spectral function (or bispectrum as it is sometimes 
called) in the analysis of certain ocean wave patterns 
see Hasselman, Munk, and MacDonald [1963]. 

We shall now briefly sketch the statement of a 
result recently obtained by Van Ness [1963] on the 
estimation of the bispectrum of a stationary process. 
Let X h EX ( =0, be a real-valued stationary process 
with jEX,<oo. Let its second and third order 
spectral functions be absolutely continuous with con- 
tinuous derivatives, the spectral density f(\) and 
bispectral density function g(\i,X 2 ) respectively 

r{t)= r e itx f(\)d\ 
n(t u tt= f f eWVvtXi, \ 2 )dhidK (15) 
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The third order moment function r z (ti, t 2 ) has the 
symmetry properties 



r 3 (*i, t 2 ) =r 3 (t 2 , U) =r 3 (—t h t 2 — t 1 ) 



(16) 



because the process X t is real-valued. Van Ness 
considers an estimate of g(\ u \ 2 ) of the form 

Pn(vi, v 2 )dvidv 2 (17) 
with p N (v u v 2 ) the third order sample moment function 

Pn(v\, V2 ^ = Jj J X t X t+lli X t + v dt (18) 

with t, t+vi, t J rv 2 restricted to the range [0, N] and 
k(v u v 2 ) a normalized (k(0, 0) = 1) continuous bounded 
uniformly integrable weight function with the sym- 
metry properties of r 3 (t u t 2 ). Here B N -^0 as N— >°° 
so that the Fourier transform of k(B N v u B N v 2 ) behaves 
asymptotically like a ^-function. Under a variety of 
boundedness and integrability conditions on the 
moment functions up to sixth order, g%{\\, X 2 ) is 
shown to be asymptotically unbiased as iV-^°° and 

lim NB 2 N gov [gUK Xa), gUK X 4 )] 

=^L/(x 1 )/(x 2 )./(x 1 +x 2 )/(x 3 )./(x 4 )/(x 3 +x 4 )] 1 ' 2 

{^(X 2 )5(X 4 )[l+2§(X 1 )][l+25(X 3 )] 
+A: 2 S(X 1 -X 3 )5(X 2 -X 4 ) 

[1+S(X I -X 4 )+45(X 1 )S(X 2 )]} (19) 

if B N ^>0, B%->c° where 0<n u M3<°°, 0<jti 2 <Mi, 
< M4 < M3 • In formula (19) 



fo= I I k 2 {v u v 2 )dv 1 dp 2 , 



x=0 



My thanks are due to J. Tukey for suggesting a 
neater way of writing the asymptotic covariance 
estimate that led to formula (19). 
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